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ABSTRACT 


The perception-action cycle is viewed within the context of research in manual control. A por- 
trait of a perception-action system is derived from the primitives of control theory in order to evalu- 
ate the promise of this perspective for what Warren and McMillan (1984) have termed “Active Psy- 
chophysics.” That is, a study of human performance that does justice to the intimate coupling 
between perception and action. 


INTRODUCTION 


Are there important differences between a human actively involved in accomplishing a goal 
directed activity and a human passively monitoring and making judgements about stimulation 
imposed from without? In the active mode the subject has control over stimulation. In the passive 
mode stimulation is controlled by an entity (generally the experimenter) other than the subject. These 
two modes may be different in terms of the control of attention; in terms of the kinds of information 
available; in terms of sensitivity to information; and are certainly different in terms of the kinds of 
activities required of the subject. Certainly Gibson’s early studies with touch suggest that active and 
passive modes are fundamentally different in the kinds of information picked up by the actor/ 
observer (Gibson, 1962). Stappers (1989) has recently shown that active control enhances visual 
form recognition. Also, research on the effects of automation on the performance of human-machine 
systems (out-of-the-loop syndrome) suggests that there are fundamental differences between systems 
where the human functions as a controller compared to systems where the human functions as a 
monitor (e.g. See Wickens, 1984, P.492). To the extent that the actor and the observer are different, 
care must be taken with how researchers generalize the results of experimentation. The domination 
of passive modes of interaction in psychological research (even in ecological research which is based 
on the concept of the perception-action cycle) may lead to inappropriate generalizations. For this 
reason a number of people (e.g. Warren & McMillan, 1984) have pointed out the need for research 
paradigms that permit subjects to actively control stimulation in pursuit of goals. In this paper, a 
tutorial review of control theory will be presented as one framework within which an “active 
psychophysics” might be pursued. 
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INPUT AND OUTPUT 


Figure 1 shows a black box representation of a human-environment system. There are two quali- 
tatively different sources of input into this black box and a single output. These inputs and outputs 
are not single dimensional entities but instead should be considered multidimensional vectors. The 
distinction between Intention and Disturbance, as qualitatively different inputs to the black box is 
critical for understanding the behavior of control systems. However, this distinction is often 
obscured in the literature on manual control. The term input is sometimes used to refer to intention 
and sometimes to disturbance (Powers, 1978). In general, a good controller will minimize the match 
between disturbance and output and will maximize the match between intention and output. In other 
words, a controller will behave so as to accomplish intentions (goals) and will do so in spite of any 
external disturbances that might perturb the system. The prototypical example is a thermostat. A 
temperature is input as an intention and this temperature is attained and maintained in spite of exter- 
nal inputs (disturbances) arising as a function of outside temperatures. 

A second qualitative distinction is important in characterizing the input signals (both intentions 
and disturbances). Inputs can be discrete or continuous. An example of a discrete input used in the 
study of human performance is the Fitts’ Law paradigm (see Jagacinski, In Press for review). The 
appearance of the target is an intentional input in which the goal of the operator is changed instanta- 
neously from one position (the home position) to a second position (the target position). Step track- 
ing is another example in which discrete signals (instantaneous changes of position) are used as 
inputs. When step tracking is performed in a pursuit mode, as illustrated in Figure 2, then the input is 
an intention. When step tracking is performed in a compensatory mode, then the input is a distur- 
bance. In discrete control paradigms, dependent measures that are often used include: 

Reaction Time - the time from the input signal onset to the onset of the response to that signal. 
This is illustrated in Figure 2. 

Movement Time - the time from the initiation of a response to the input signal to the completion 
of the response (e.g., target capture). 

Accuracy - the match between intention and action (output) at the end of a response sequence. 

Submovements - often the output resulting from a discrete input can be parsed into segments 
(e.g., submovements). Important measures include the number of submovements; the duration of 
individual submovements; the accuracy of individual submovements; the peak velocities; and the 
peak accelerations. 

Continuous signals can also be used as input to the black box. Typically, the continuous signals 
used in manual control experiments are constructed as a sum of sine waves. There are two reasons 
for this choice. First, Fourier’s Theorem shows that any periodic signal can be approximated as a 
sum of sine waves. Thus, sine waves are fundamental building blocks for constructing a wide range 
of signals. A second reason for using sine waves to construct signals is that for a linear servomecha- 
nism a sine wave input will result in a sine wave output at the same frequency, but changed in ampli- 
tude and phase. The pattern of amplitude and phase changes can be extremely useful for drawing 
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inferences about the nature of the black box (e.g., the transfer functions). Also, frequency can be 
used as a signature to differentiate the sensitivity of the black box to various kinds of inputs. The use 
of frequency signatures to differentiate sensitivity will be discussed further in a later section of the 
paper. When continuous signals are input as intentions, then the subject’s task is called a pursuit 
tracking task. In this task the subject sees both a continuously changing target (e.g., a roadway) and a 
cursor representing her position with respect to the roadway. A good controller would be one that 
minimized deviations between her position and target position. When continuous signals are input as 
disturbances, then the subject’s task is called a compensatory tracking task. Here the subject’s goal is 
a fixed position (e.g., center of screen or constant altitude) and a disturbance (e.g., windgust) is input 
that drives the subjects away from their fixed goal. In pursuit tracking, subjects can see movements 
of the goal and movements of themselves with respect to that goal. In compensatory tracking, sub- 
jects see only their own movement with regard to the fixed goal. For research using continuous 
inputs the dependent variables typically used include: 

RMS Error - this is the square root of the sum of squared deviations between cursor (ego or 
vehicle) position and the goal position (summed over samples) divided by the number of samples. 
This method of scoring results in a differential weighting of small and large errors. 

Small errors contribute proportionally less to RMS error than do large deviations. 

RMS Control and RMS Control Velocity - these measures are similar to RMS error. They are 
indexes of the amount of control activity. 

Time-on-Target (TOT) - this is a measure of the proportion of time during a tracking trial that 
the subject is within the boundaries of the target. 

Am plitude and Phase - the amplitude and phase are measured at each frequency of input. The 
ratio of amplitude in the output to amplitude in the input signal is termed gain. These measurements 
are important for characterizing the transfer function of the black box. 

Remnant - the remnant is the output power at noninput frequencies. This is an index of the 
control variance that is not correlated with input signals. 


NEGATIVE FEEDBACK CONTROL 


A simple system that acts to attain and maintain an intention in spite of disturbances is a negative 
feedback system. Figure 3 shows a simple negative feedback device. The new ingredient that the 
negative feedback system introduces is error. This is the difference between the intention or goal and 
the current state of the system. A negative feedback system is driven by error, that is, when error is 
zero there is no action in this system. When error is non-zero this system will attempt to reduce the 
error. Whether or not the system is successful in reducing error will depend on the characteristics of 
Q. Figure 3 shows a derivation of the relation between Intention, Disturbance, and Output as medi- 
ated through Q. The equation relating these elements is: 
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[G/(1+G)j* Intention + [l/(l+G)]*Disturbance = Output 


( 1 ) 


Note from Equation 1 that if £ is a simple multiplier then the greater the value of £ (i.e., the higher 
the open loop gain) the closer will be the match between Output and Intention. The term that oper- 
ates on Intention will go to 1 as £ becomes large. The term that operates on Disturbances will go to 

0 as £ becomes large. Thus, as £ becomes large Equation 1 will reduce to: 

Intention = Output (2) 

In nature £ is never a simple multiplier. For all physical systems there will be a delay associated 
with £. For control purposes it is not the absolute time associated with this delay but the time rela- 
tive to the frequency of the signal. That is, the key dimension will be the proportion of a cycle that a 
signal is delayed. This is termed phase lag. If a signal is delayed by 180 degrees then the negative 
feedback system will result in a diverging error. Such a system is said to be unstable. For good con- 
trol £ should have high gain when the phase lag is less than 180 degrees. The higher the gain, the 
faster error will be reduced. £ should have low gain, less than 1, as the phase lag approaches and 
exceeds 180 degrees. This relation between gain and time delay is illustrated in Figure 4, which is 
adapted from Jagacinski (1977). The graph shows three regions sluggish control, good control, and 
unstable control. If the time delay is small (small phase lag) and the gain is low then error will be 
reduced very slowly. An example of a sluggish response to a step input is shown in Figure 4. If the 
time delay is large and gain is high the error will not be reduced and in fact will become greater. This 
is the region of unstable control. Pilot induced oscillations in flight result from a pilot responding 
with two high a gain given the time delays associated with the system. An example of an unstable 
response to a step input is also shown in Figure 4. If gain is high and time delay is small or if gain is 
low when time delay is large then good tracking will result. Two examples of the response of a good 
tracker to a step input are illustrated in Figure 4. Note that as the time delay becomes greater the 
range of gains that will result in good tracking diminishes. 

The relationship between gain and phase lag can also be illustrated using a Bode plot. The Bode 
plot shows open loop gain (in decibels) and phase lag (in degrees) plotted as a function of the log of 
frequency (in radians/sec). Figure 5 shows the pattern of gain and phase lag that would be obtained 
for a good controller. This pattern represents good control in that for those frequencies with phase 
lag less than 180 degrees gain is high. Thus, intentional signals at those frequencies will be followed 
closely in the output and disturbances at those frequencies will be filtered out (will not show up as 
output). In other words, errors will be eliminated quickly. For those frequencies with phase lags 
greater than 180 degrees gain is less than 1. Thus, the system will be stable. Intentional signals at 
those frequencies will not be followed in the output and disturbances at those frequencies will not be 
filtered out (they will be part of the output). 

A key landmark in the Bode plot is the “crossover point,” the point at which gain is equal to 

1 (0 db). For the system to be stable the phase lag must be less than 180 degrees at that point, the 
distance of the phase lag from 180 degrees is called the phase margin of the system. A positive phase 
margin is required for stable control. The frequency of the crossover point indicates the bandwidth of 
the controller. Intentional signals at frequencies below the crossover point will be represented in the 
output. Intentional signals at frequencies above the crossover point will be filtered out (will be 
attenuated in the output). 
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A final point to be noted about negative feedback, closed-loop systems concerns the concept of 
time. The common sense notions of before and after do not apply. Errors do not precede actions 
which in turn precede feedback. Errors, action, and feedback are continuously available. In place of 
the common sense notion of time is the concept of phase. Action can be in-phase with feedback 
(perception) or out-of-phase. When in-phase the system will be stable. When sufficiently out-of- 
phase the system will be unstable. 


MANUAL CONTROL 


Manual control is the study of negative feedback control systems in which the loop is closed 
through a human operator. That is, the human operator is given a task or goal to accomplish this goal 
is accomplished by observing displays and manipulating controls. This situation is illustrated in Fig- 
ure 6. Note that the G in the forward loop of Figure 4 has been replaced by two boxes in the forward 
loop of Figure 6. One box, labelled Controller, represents the transfer function for the human opera- 
tor. The second box, labelled Plant, represents the transfer function for the physical system that the 
human is interacting with (e.g., dynamics of the helicopter). The central problem for a theory of 
manual control has been to build a model or theory of the human operator. Two approaches to mod- 
eling the human will be distinguished. One approach assumes that the human operator responds con- 
tinuously to error. The second approach assumes that the human responds in a discrete fashion. 

Continuous Control 

Early researchers began with the assumption that the transfer function of the human operator 
would be invariant, independent of the plant dynamics. It was assumed, that once this transfer func- 
tion was discovered it could be used to predict performance across a wide range of plant dynamics. 
McRuer and his colleagues (e.g., McRuer & Jex, 1967; McRuer & Krendel, 1974; McRuer & Weir, 
1969) soon discovered that this definitely was not the case. As the dynamics of the plant changed, so 
to, did the describing function for the human operator. The invariant, as McRuer et al. discovered 
was not at the level of the human but was at the level of the total forward loop (human + plant) 
describing function. This invariant at the level of the human/plant combination was the basis for the 
classic “crossover” model. The key insight behind the crossover model is illustrated in Figure 7. The 
first column in Figure 1 shows Bode diagrams and transfer functions [using Laplace notation] for 
three simple plant dynamics. The second column in Figure 7 shows describing functions obtained for 
humans controlling each of the three dynamic plants. The final column shows the describing func- 
tion for the human/plant combination. Note that the patterns in Column 3 are invariant and that they 
have the same form as the “good” controller illustrated in Figure 5. What was surprising to earlier 
researchers should be obvious in retrospect. The constraints on good stable performance operate at 
the level of the total forward loop (human + plant). To do the task the human must operate within 
those constraints and therefore must adapt to the plant dynamics in a way that is consistent with 
those constraints. Thus, the “crossover” model predicts that in the region of crossover the human 
plus the plant will approximate the transfer function shown in Column 3 of Figure 7. 

In adjusting to the plant dynamics to both satisfy the demands to minimize RMS error and to 
satisfy the constraints for stability the human behaves like an optimal controller. This observation 


125 



was the basis for the “optimal control” model of the human operator (e.g. Baron & Kleinman, 1969; 
Kleinman, Baron, & Levison, 1970; Kleinman, Baron, & Levison, 1971). The optimal control model 
assumes that the human operator uses an internal (mental) model of the plant dynamics to estimate 
the current states of the system from delayed, noisy observations of display position and velocity. 
The human responses to these states are based on an optimal control law which chooses response 
gains that minimizes a linear combination of squared tracking error and squared control velocity. 
Thus, in a sense, the model assumes that the operator attempts to achieve minimum error with mini- 
mum effort. These responses are filtered through the limb dynamics and are contaminated by motor 
noise. 

The optimal control model has been popular because there is a natural mapping from the ele- 
ments of the model to the stages (encoding, estimation, decision, response) of the standard informa- 
tion processing model that has dominated modem psychology (See Pew & Baron, 1978). The opti- 
mal control model also provides a better fit over a wider range of frequencies to human performance 
data than does the crossover model. However, to do so it requires a greater number of parameters. 

The crossover model and the optimal control model both assume that the human responds in a 
continuous, proportional (linear) fashion to error and error velocity. However, there is much evi- 
dence that the human is not linear (e.g. see Knoop, 1978). For example, there is the presence of rem- 
nant in the human control response. Remnant is power at output frequencies not present in the input. 
As noted in an earlier section, a linear system would only have output at the input frequencies. The 
optimal control model accounts for the remnant by assuming the presence of broad band white noise 
injected by human perceptual and motor processes. The non-white shape of the measured remnant is 
thought to reflect the dynamics of the humans’ perceptual and motor processes. Others have argued 
that the remnant arises, at least in part, due to the discrete, nonlinear nature of the human transfer 
function. 


Discrete Control 

In discussing discrete control models of the human operator three classes of models will be pre- 
sented — synchronous discrete controllers, asynchronous discrete controllers, and hierarchical 
controllers. 

Bekey (1962) lists a number of studies that have found evidence of a “psychological refractory” 
period when a human is required to respond to discrete stimuli spaced by less than about 0.5 seconds 
(Hick, 1948; Welford, 1952; Davies, 1957). One inference that might be drawn from this finding is 
that the human “acts on discrete samples of information from the external world.” Figure 8, adapted 
from Bekey (1962) gives examples of two synchronous discrete controllers. These controllers act on 
discrete observations taken at a fixed frequency. A synchronous sampler with a 0-order hold 
responds as a function of the position observed at each sample. The synchronous sampler with a 1st- 
order hold responds as a function of the position and velocity observed at each sample. Three impor- 
tant attributes of synchronous discrete controllers noted by Bekey (1962) are: 

(1) Changes in the input cannot have any effect until the next sampling instant occurs. 
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(2) The presence of the sampler limits the frequencies which can be reconstructed at its output 
to those not exceeding one-half the sampling frequency. 

(3) The action of the sampler generates harmonics in the output which extend over the entire 
frequency spectrum, even when the input is band limited. (Bekey, p. 45-46.) 

The last attribute provides an alternative explanation for the remnant power routinely observed in 
human tracking data. 

A synchronous discrete controller responds at a fixed frequency. An asynchronous controller 
responds at irregular intervals. Angel & Bekey (1968) have proposed a finite-state model for manual 
control that behaves asynchronously. The logic of the finite-state controller is illustrated in Figure 9. 
Inputs to this controller are coarsely quantized with regard to threshold boundaries on position and 
velocity. These boundaries are the dashed lines in Figure 9a. These quantized inputs are responded to 
with simple force time programs which are shown in each region of state space. For example a large 
position error with low velocity evokes a large amplitude bang-bang response. This type of model 
has great intuitive appeal for modeling human control of second-order control systems, where there 
is evidence that humans exhibit bang-bang control (Young & Meiry, 1965). This nonlinear style of 
control provides still another alternative explanation for remnant. 

Costello (1968) proposed a model of the human tracker using a hierarchical control model. 
Costello’s model is illustrated in Figure 9b. Costello proposed two modes of control. He proposed 
that the human controller responded to small errors and error velocities in a manner consistent with 
the crossover model. This is the central region of the state space identified with the constant coeffi- 
cient mode. To large errors, the model predicts that the human will respond in a time optimal bang- 
bang fashion. Costello called this the surge mode. Jagacinski, Plamondon, and Miller (1987) have 
also employed a multi-level style of modeling in which a number of low level motion generators 
(Herding mode, predictive mode, close following mode, fast acquisition mode) are combined with 
finite state logic to model human performance in capturing evasive, moving targets. 


SUMMARY 


The continuous control models have dominated much of the work on manual control. These 
models have been useful tools for evaluating human control systems and for making predictions 
about stability of these systems. They have particularly been widely used for studying vehicular con- 
trol. However, it is clear that some of the assumptions made by these models must be questioned. 
One must wonder whether the practical utility and success of these models has retarded scientific 
progress in understanding human control. 

There is one intervening variable that should be considered when choosing between the linear, 
proportional control models (i.e., crossover, optimal control, synchronous controller) and the nonlin- 
ear, discrete control models (i.e. asynchronous or hierarchical controllers). That is the time lag of the 
physical system being controlled. The linear, proportional control models work well for systems that 
have small time lags (e.g., high performance aircraft). However, these types of models are totally 


127 



inadequate for systems with long time lags such as thermodynamic systems (see Crossman & Cooke, 
1974). For slow responding systems it is clear that humans respond in a discrete, nonproportional 
fashion. 

This has been a very brief and selective review of some of the models that have been proposed 
for the human controller. For the most part, the research that has inspired these models has employed 
simple laboratory tracking tasks using moving cursors on CRT displays. In this kind of task the error 
signal is clearly defined and thus the perceptual problems have not generated very interesting prob- 
lems. It remains for an ecological psychology to study control behavior with less well defined error 
displays (e.g., optical flow fields). This review is presented here because as the perceptual problems 
are addressed, our ability to draw correct inferences about perception will depend on our use of 
informed assumptions about action. 

Closing the Loop Through the Optic Array 

“...instead of searching for mechanisms in the environment that turn organisms into trivial 
machines, we have to find the mechanism within the organisms that enable them to turn their envi- 
ronment into a trivial machine.” (von Foerster, 1984, p. 171) 

The laboratory tracking task, in one sense, is a task that turns humans into a trivial machine (e.g. 
a simple gain, integrator, or differentiators). The error signal and the goal of the operator are “trivial” 
relative to the signals by which humans control their own locomotion. The problem in more natural 
environments is not simply to generate the appropriate control law, but to extract from the “booming, 
buzzing confusion” the information that specify die goals and the error with respect to those goals. 
Gavan Lintem (personal communication) has observed that, when learning to fly, controlling the air- 
plane (getting it where you wanted it) was not the problem. The problem was knowing where you 
wanted to be. That is knowing what the correct glideslope looked like. A critical aspect of the organ- 
ism turning its environment into a trivial machine may be an ability to pick-up information about 
regularities in the environment. Thus, it is the tuning to invariants in perceptual arrays that allows the 
“booming, buzzing confusion” to be managed. How information (i.e. invariants, constraints, or 
structure) in the optic array supports action has been a central question for ecological psychology 
ever since Gibson, Olum, and Rosenblatt’s (1955) classic analysis of parallax and perspective during 
aircraft landings. However, in asking questions about the pick-up of information from optic arrays 
there is little evidence of a commitment to “active vision.” Many of the studies of information pick- 
up have employed passive psychophysical methodologies (e.g., Warren, 1976; Owen, Warren, 
Jensen, Mangold, and Hettinger, 1981; Cutting, 1986; Anderson and Braunstein, 1985; Warren, 
Morris and Kalish, 1988; Larish and Flach, in press). Not only have our experiments employed pas- 
sive tasks, but Stappers, Smets, and Overbeeke (1989) have argued that our conceptualizations of the 
flow field and of the information within it have been founded on the image of a passively translating, 
disembodied eye. They argue that many of the classic ambiguities disappear when one considers the 
information in optic flow fields generated by bouncing eyes locomoting over a surface of support. 
Stappers, et al. note that formal accounts of optic flow (e.g. Longuet-Higgins and Prazdny, 1980; 
Koenderink, 1986) “neglect the fact that the optic flow is largely brought about by the actions of the 
observer, and for just this reason it can be relative to the observer’s effectivities: the observer’s 
actions scale the information he samples.” 
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The Performatory Loop 


Figure 10 illustrates an initial framework for asking questions about the perception-action cycle 
where the loop is closed through an optic array. In this framework, the human observer is given an 
implicit (e.g. maintain stable posture) or explicit (e.g. maintain a constant altitude) goal. Control 
activity is then measured as a function of manipulations of the optic array (e.g. front vs. side view, 
l ame llar vs. radial flow, parallel vs. perpendicular texture). A number of studies have begun to 
appear that have been framed in this manner. Stoffregen (1985) and Andersen and Dyer (1989) have 
used postural regulation as a control problem within which to study optic flow. Owen and Warren 
(1987) report research that examined control responses to discrete changes in acceleration and to 
ramp changes in altitude in order to identify the optical information that specifies egospeed and alti- 
tude. Warren (1988) review a series of studies that have examined altitude control with a continuous, 
sum-of-sines disturbance. Within this framework, Warren has varied the nature of the optic array 
(e.g. presence of perspective roadway) and the nature of the task (e.g. altitude maintenance, or fly as 
low as possible) in order to isolate the functional optical information for altitude. Johnson, Bennett, 
O’Donnell, and Phatak (1988) have also used an altitude regulation task to examine the utility of 
alternative structures in the optic array. 

The Johnson et al. paper is particularly useful for illustrating the promise of control theoretic 
methodologies for supporting inferences about perception and action. In order to highlight the logic 
of the control methodologies the details of the experiment will be greatly simplified. Johnson et al. 
were interested in comparing the relative efficacy of two sources of optical information about alti- 
tude — splay angle and optical density. To address this question displays were chosen which isolate 
the two sources of information. These are shown in Figure 11a. Texture parallel to the direction of 
travel contains splay. Texture perpendicular to the direction of travel contains optical density but no 
spay. Square texture combines both splay and optical density. Crossed with the type of display were 
three types of disturbance. A horizontal disturbance (altitude) affected both parallel (splay) and per- 
pendicular (optical density) texture. A fore-aft disturbance (headwind) affected only perpendicular 
texture. Finally, a lateral (side-to-side) disturbance affected only parallel texture. The three distur- 
bances were constructed from sine waves so that the bandwidths of the disturbances were similar, 
but so that the frequencies were specific to a disturbance (no shared harmonics). This is illustrated in 
Figure 1 lb. Frequency can now be used as a signature to identify the control activity specific to opti- 
cal features. Johnson et al. found better control of altitude with perpendicular texture. They also 
found that there was more altitude control resulting from the fore-aft disturbance (seen only in per- 
pendicular texture), than from the lateral disturbance. This provides strong evidence that for the 
hover task studied, perpendicular texture provided a powerful source of information, guiding altitude 
control behavior whether it was specific to altitude or not. 

Exploratory Behavior 

The framework in Figure 10 represents an advance over passive psychophysics. However, 
experiments designed within that framework, still constrain the human to behave as a rather simple 
machine (servomechanism). In the framework of Figure 10 behavior arises only as a function of 
error with respect to performatory goals. However, humans act, not only to accomplish performatory 
goals, but also, humans act to pick-up information. Humans actively explore the environment. This 
exploratory mode of behavior is intimately coupled with performatory modes of behavior. 


129 



Information picked-up through exploratory activity will often support performatory activity. Also, 
performatory activity will itself result in the pick-up of information. An important challenge for an 
active psychophysics will be the study of the coupling of performance with exploration. Experimen- 
tal paradigms must include tasks that allow or even encourage exploration. Active psychophysics 
must explore measurement and analysis techniques for parsing exploratory and performatory activi- 
ties; or must discover meaningful higher-order parameters for gauging the interaction of exploratory 
and performatory modes. 

One basis for parsing exploratory and performatory activities might be the distinction between 
correlated and uncorrelated power resulting from frequency analyses of control behaviors. For sys- 
tems with small time constants and for well trained operators it might be expected that performatory 
activities will be closely linked to the “driving function” (i.e., the changing goal or the disturbance 
that perturbs the system from a fixed goal). Thus, performatory activity will be task driven. Explor- 
atory activity, however, originates with the operator. This will likely be uncorrelated with the driving 
function and therefore, will appear as remnant. As we have seen earlier in this paper exploratory 
activity will probably not be the only source of remnant. Other sources that have been considered 
include perceptual/motor noise, discrete response strategies, nonlinearities, and uncorrelated optical 
activity. Remnant appears to be rich in information about the human operator. In fact, it could be 
argued that most of the psychology resides in the remnant. Whereas the correlated power carries 
little information about the operator, informing us, rather about the task. 

Higher order parameters for gauging the interaction of performatory and exploratory modes 
might be stability and bandwidth. As operators discover more effective ways to pick-up information, 
this should be reflected in either larger stability margins or in greater bandwidths. 

Questions about remnant may be the only avenue for addressing the performatory/exploratory 
distinction within the experimental framework shown in Figure 10. In this framework there is only a 
single response channel for both exploratory and performatory activities. Frequency analysis is a 
useful tool for partitioning different signals within a single channel. It may be easier to study per- 
formatory/exploratory interactions if our experimental framework is expanded to permit a second 
channel of activity. A natural choice for this second channel of activity would be eye movements as 
shown in Figure 12. 

While it is not impossible to imagine situations where eye movements can have a performatory 
function (e.g., social interactions), in many natural task situations eye movements are purely explor- 
atory. That is, they have no direct effect on error with respect to performatory goals. TTie indirect 
effects, however, may be great in terms of the information pick-up that the eye movements mediate. 
For this reason, the study of eye movements must be a critical element within an active 
psychophysics. 

When the possibility of eye movements is introduced an important theoretical question must be 
addressed. This involves the question of whether information is specific to an ambient optic array or 
to the retinal array. For example, the focus of expansion (Gibson, 1947; 1950; 1958; see also 
Warren, Morris, and Kalish, 1988) is an invariant that specifies the direction of locomotion which 
has been defined relative to the ambient optic array. That is, the focus of expansion is a pattern 
within optic flow that arises as a consequence of a moving observation point. This pattern is a 
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consequence of ecological optics — the properties of light. It is independent of the nature of a sensory 
mechanism (e.g. simple vs multifaceted lens) and is independent of the viewport (i.e., where the 
organism is looking). On the other hand. Cutting’s (1986) differential motion parallax has been pro- 
posed as an alternative invariant specifying direction of locomotion that has been defined with 
respect to the retinal array. That is, the invariant relations of differential motion parallax are specific 
to a viewpoint. They depend on a particular point of fixation. 

I assert that both the ambient optic array and the retinal array descriptions have an important 
place in an active psychophysics. The world (including the observer) structures the ambient array. 
The structure in the ambient array is information about the world and the observer. This structure is 
present at a station point and in the relations between station points. Pick-up of information requires 
first a transducer sensitive to the energy that carries the structure. Second, pick-up depends upon 
activity (sampling). What information is picked up depends on the activity of the observer? A sta- 
tionary observer can pick up only the information at a single station point. This is an extremely 
impoverished view. A moving observer has access to information from multiple station points and 
has access to the information in the relations across station points. Note that no information about 
environmental layout is created by movement. The information exists whether the observer moves or 
not. Movement simply makes the information available. Also note, that a particular movement only 
provides access to the information at the station points sampled and in the relations across those sta- 
tion points. Some ways of acting will reveal different information than others. Therefore, some ways 
of acting will be more effective for certain tasks, because the information made available will be 
more appropriate. 

An important challenge for an active psychophysics will be to provide a framework for evaluat- 
ing the effectiveness of sampling behavior. The challenge is not to provide an absolute metric for 
effectiveness, because effectiveness can only be measured relative to a task, but to provide a collec- 
tion of methodologies for asking questions and drawing inferences with regard to sampling behavior. 
Thus, it is meaningful and important to ask the following question: For a given pattern of sampling 
behavior what information is in principle available to the actor/observer? This is where the retinal 
array becomes important. The retinal array is one kind of record of the information made available 
by a particular pattern of sampling behavior. 

Mathematical descriptions of the retinal array can be very useful for generating hypotheses about 
what subset of the information from the ambient array is preserved over a particular set of samples 
from that array. However, it is important to note that there is an asymmetry in the logic of mathemat- 
ical descriptions of both the ambient field and the retinal field. If an invariant mathematical relation- 
ship can be demonstrated between structure in the ambient array (or structure on the retina) and 
properties of the world (including observer) then this is proof that information is present. However, 
failure to discover a mathematical relationship does not prove that there is no structure. In this sense, 
no particular form of mathematical representation is privileged. 

An active psychophysics must appreciate the importance of mathematical analyses of the ambi- 
ent array and of the retinal array. However, it should never be constrained by these analyses. These 
mathematical analyses will help us to discover what are interesting questions to ask. However, the 
answers can only come from observations of behavior. 
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For an active psychophysics to be complete, observations must be made in which the actor has 
unrestricted and independent control over performatory and exploratory modes of behavior. In all of 
the studies cited above that examined control through optic arrays, performatory and exploratory 
behavior were constrained so that the actor could only look where he was going. However, in most 
natural environments no such restriction is present. When given independent control of exploratory 
behavior, where do people look? Are some patterns of looking more or less effective than others? Do 
different pattens of looking result in qualitatively different styles of control? These are the kinds of 
questions that motivated Gibson’s (1962) observations on active touch (see also Stappers, 1989). 
These kinds of questions must be central to an active psychophysics. 

Adaptation and Learning 

Adaptation and learning are obvious and important side effects of the interaction between per- 
formatory and exploratory modes of behavior. Exploratory activity results in the discovery of infor- 
mation. The more information available to the actor the greater will be the number of control strate- 
gies that are available. A wider range of control strategies will open the possibility for both greater 
precision of control and greater stability. Figure 13 shows the addition of “adaptive logic” to our 
growing diagram of a perception/action cycle. Behind this small box hides enough mysteries to 
support many careers in Psychology. 

The signals entering the adaptive logic box are of the same general nature as the signals through- 
out the network. These signals are patterns of energy in space-time and these signals are operated on 
by the boxes in the diagram. However, the signals leaving the adaptive logic box are different. They 
represent operators that operate on the other boxes. For example, output from the adaptive logic box 
may result in a change of the transfer function between observation of error and action. This results 
in an interesting circularity or coupling. The patterns of energy in space-time (connections between 
boxes) are both operators and operands. So to, the embodied constraints represented as boxes are 
themselves operated on by the very signals upon which they operate. This kind of coupling between 
system and signal is also seen in neural nets and connectionist machines that tune to invariants in 
stimulation (see McClelland and Rumelhart (1986) for review). 

Control theoretic technologies may not be the most useful tools for organizing our thinking with 
regard to this coupling of system and signal. Field descriptions such as those described by Kugler 
and Turvey (1987) may be more useful. However, as we explore new modes of description we 
should proceed armed with the intuitions of those who have gone before. McRuer, Allen, Weir, and 
Klein (1977) have proposed the Successive Organization of Perception (SOP) model as a tool for 
understanding how the control logic might change with learning. This model, shown in Figure 14, 
includes three modes of tracking. The compensatory mode has been discussed throughout this paper. 
In this mode the human acts like a servomechanism responding to error between intention and out- 
put. The compensatory mode would dominate for a naive operator. As the operator becomes experi- 
enced he begins to learn the dynamics of the plant being control. Thus, he can anticipate the response 
of the plant. This allows him to respond directly to intentions rather than to error. To the extent that 
his anticipations are incorrect the residual error will be reduced as a result of the inner compensatory 
loop. If the environment that specifies the intention behaves in a consistent way (e.g. a track com- 
posed of a single sine wave), then the observer may tune to these consistencies. In other words, the 
operator may learn the “rule” or “pattern” that governs the input. This will allow a response to the 
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higher order pattern and free the operator from the requirement to continuously monitor input or 
error. This mode has been called precognitive. For example, an operator tracking a cursor driven by 
a single sine wave, may synchronize his response to the periodic pattern. Thus, the operator could 
close his eyes and still maintain close tracking (at least for short periods). While one mode or the 
other may dominate, depending on the state of the operator (e.g. experience level) or the state of the 
task (e.g. regularity), all modes are expected to operate in concert complementing each other. 

Important empirical work has also been done on adaptation in the context of manual control (e.g. 
Young, 1969; Wicken, 1984). This empirical work should be instructive to those pursuing an active 
psychophysics. The following challenge from Young (1969) signifies the need for an active psy- 
chophysics to organize our thinking with regard to adaptive control. 

“...what is being offered to solve the manual control problems of tomorrow? What will be the 
“critical task” facing the astronaut entering the atmospheres of a strange planet, the captain of an 
SST, the pilot of a commercial airliner making an approach in zero-zero visibility, the VTOL pilot 
guiding his unstable vehicle to a downtown landing field, the submarine commander, or the engineer 
on a high speed transportation system? Will they be involved in compensatory tracking? Obviously 
not. They will be on board for the versatility, adaptability, and reliability they add to an automatic 
system. They will be expected to observe the environment and use “programmed adaptive control” 
to change plans. They will monitor instruments and repair malfunctioning components. They will 
control in parallel with the automatic system and take over in the event of failure. What is the extent 
of the theory for predicting man-machine behavior in these simulations? It is almost nil.” (Young, 
1969, p. 329) 


CONCLUSIONS 


“The world is as many ways as it can be truly described, seen, pictured, etc. and there is no such 
thing as the way the world is.” Nelson Goodman (1968) 

Figure 13 represents one way to picture a perception/action cycle. It is not the way to picture per- 
ception/action cycles. The representation is not a roadmap for the future. In fact, it could be argued 
that if the representation in Figure 13 is taken too literally, then it will severely constrain our think- 
ing and will be an obstacle to future progress. If the representation in Figure 13 is useful it is as a 
map to the past. That is, as a link to the study of manual control. The research on manual control has 
much to offer to anyone interested in the coupling of perception and action. As a new active psy- 
chophysics is molded, its shape should not be constrained by the cybernetic hypotheses that guided 
much of the work in manual control. However, our vision of the future of active psychophysics will 
be much clearer if we stand on the shoulders of those who have gone before. The methodologies of 
manual control offer an important alternative to the passive methodologies that dominate current 
psychophysics. If these methodologies are applied with caution and restraint, the future of an active 
psychophysics will hold great promise. Alternatively, the challenges posed by an ecological 
approach to perception and action promise to rejuvenate an area of research that is being lulled to 
sleep reliving past successes. 
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Disturbance 



Figure 1 . A black box representation of a human-environment system. 
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Figure 2. Responses to a step input on the intention channel (pursuit) and on the disturbance chan- 
nel (compensatory). 
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Figure 3. A simple negative feedback system. 
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Figure 4. (A) Illustrates tracking quality as a function of gain (sensitivity to error) and the time 
delay (delay of feedback) (Adapted from Jagacinski, 1977) (B) Illustrates responses to step inputs for 
the regions shown in A. 
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Figure 5. A Bode plot typical of a “good” controller. 
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Figure 6. The manual control framework. 
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the demands imposed by three simple plants. 









Figure 8. Two strategies for discrete synchronous control. Zero-order extrapolates based on posi 
tion. First-order extrapolates based on position and velocity (Adapted from Bekey, 1962). 
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Figure 9. (A) Logic for an asynchronous discrete controller proposed by Angel and Bekey (1968) 
(B) Logic for hierarchical “surge” model proposed by Costello (1968). 







Figure 10. Closing the loop through the optic array. 
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Figure 11. Illustrates logic of approach employed by Johnson et al. (1988) to evaluate alternative 
invariants for altitude control (A) Parallel (splay) texture, perpendicular (density) texture, and square 
texture (B) Frequency is used as a signature to isolate the effects of three disturbances (altitude, head 
wind, lateral) that were chosen because of their specific impacts on parallel and perpendicular 
texture. 
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Figure 12. Uncoupling the eye (exploratory mode) from the hand (performatory mode). 
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Figure 13. Adaptation — operating on operators. 
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Figure 14. The Successive Order of Perception model (SOP) proposed by McRuer et al. ( 1977) 
includes three control modes (a) compensatory, (b) pursuit, and (c) precognitive. 
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